{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 项目：整理Netflix电影演员评分数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 分析目标"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此数据分析的目的是，整理不同流派影视作品，比如喜剧片、动作片、科幻片中，各演员出演作品的平均IMDB评分，从而挖掘出各个流派中的高评分作品演员。\n",
    "\n",
    "本实战项目的目的在于练习整理数据，从而得到可供下一步分析的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 简介"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原始数据集记录了截止至2022年7月美国地区可观看的所有Netflix电视剧及电影数据。数据集包含两个数据表：`titles.csv`和`credits.csv`。\n",
    "\n",
    "`titles.csv`包含电影及电视剧相关信息，包括影视作品ID、标题、类型、描述、流派、IMDB（一个国外的在线评分网站）评分，等等。`credits.csv`包含超过7万名出现在Netflix影视作品的导演及演员信息，包括名字、影视作品ID、人物名、演职员类型（导演/演员）等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`titles.csv`每列的含义如下：\n",
    "- id：影视作品ID。\n",
    "- title：影视作品标题。\n",
    "- show_type：作品类型，电视节目或电影。\n",
    "- description：简短描述。\n",
    "- release_year：发布年份。\n",
    "- age_certification：适龄认证。\n",
    "- runtime：每集电视剧或电影的长度。\n",
    "- genres：流派类型列表。\n",
    "- production_countries：出品国家列表。\n",
    "- seasons：如果是电视剧，则是季数。\n",
    "- imdb_id：IMDB的ID。\n",
    "- imdb_score：IMDB的评分。\n",
    "- imdb_votes：IMDB的投票数。\n",
    "- tmdb_popularity：TMDB的流行度。\n",
    "- tmdb_score：TMDB的评分。\n",
    "\n",
    "`credits.csv`每列的含义如下：\n",
    "- person_ID：演职员ID。\n",
    "- id：参与的影视作品ID。\n",
    "- name：姓名。\n",
    "- character_name：角色姓名。\n",
    "- role：演职员类型，演员或导演。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导入数据分析所需要的库，并通过Pandas的`read_csv`函数，将原始数据文件\"titles.csv\"里的数据内容，解析为DataFrame并赋值给变量`original_titles`。将原始数据文件\"credits.csv\"里的数据内容，解析为DataFrame并赋值给变量`original_credits`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "original_titles = pd.read_csv(\"titles.csv\")\n",
    "original_credits = pd.read_csv(\"credits.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ts300399</td>\n",
       "      <td>Five Came Back: The Reference Films</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>This collection includes 12 World War II-era p...</td>\n",
       "      <td>1945</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>51</td>\n",
       "      <td>['documentation']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.600</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>['drama', 'crime']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>['drama', 'action', 'thriller', 'european']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>['fantasy', 'action', 'comedy']</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>tm120801</td>\n",
       "      <td>The Dirty Dozen</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>12 American military prisoners in World War II...</td>\n",
       "      <td>1967</td>\n",
       "      <td>NaN</td>\n",
       "      <td>150</td>\n",
       "      <td>['war', 'action']</td>\n",
       "      <td>['GB', 'US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0061578</td>\n",
       "      <td>7.7</td>\n",
       "      <td>72662.0</td>\n",
       "      <td>20.398</td>\n",
       "      <td>7.600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                title   type  \\\n",
       "0  ts300399  Five Came Back: The Reference Films   SHOW   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "4  tm120801                      The Dirty Dozen  MOVIE   \n",
       "\n",
       "                                         description  release_year  \\\n",
       "0  This collection includes 12 World War II-era p...          1945   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "4  12 American military prisoners in World War II...          1967   \n",
       "\n",
       "  age_certification  runtime                                       genres  \\\n",
       "0             TV-MA       51                            ['documentation']   \n",
       "1                 R      114                           ['drama', 'crime']   \n",
       "2                 R      109  ['drama', 'action', 'thriller', 'european']   \n",
       "3                PG       91              ['fantasy', 'action', 'comedy']   \n",
       "4               NaN      150                            ['war', 'action']   \n",
       "\n",
       "  production_countries  seasons    imdb_id  imdb_score  imdb_votes  \\\n",
       "0               ['US']      1.0        NaN         NaN         NaN   \n",
       "1               ['US']      NaN  tt0075314         8.2    808582.0   \n",
       "2               ['US']      NaN  tt0068473         7.7    107673.0   \n",
       "3               ['GB']      NaN  tt0071853         8.2    534486.0   \n",
       "4         ['GB', 'US']      NaN  tt0061578         7.7     72662.0   \n",
       "\n",
       "   tmdb_popularity  tmdb_score  \n",
       "0            0.600         NaN  \n",
       "1           40.965       8.179  \n",
       "2           10.010       7.300  \n",
       "3           15.461       7.811  \n",
       "4           20.398       7.600  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_titles.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>character</th>\n",
       "      <th>role</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3748</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Robert De Niro</td>\n",
       "      <td>Travis Bickle</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14658</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Jodie Foster</td>\n",
       "      <td>Iris Steensma</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7064</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Albert Brooks</td>\n",
       "      <td>Tom</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3739</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Harvey Keitel</td>\n",
       "      <td>Matthew 'Sport' Higgins</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>48933</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Cybill Shepherd</td>\n",
       "      <td>Betsy</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   person_id       id             name                character   role\n",
       "0       3748  tm84618   Robert De Niro            Travis Bickle  ACTOR\n",
       "1      14658  tm84618     Jodie Foster            Iris Steensma  ACTOR\n",
       "2       7064  tm84618    Albert Brooks                      Tom  ACTOR\n",
       "3       3739  tm84618    Harvey Keitel  Matthew 'Sport' Higgins  ACTOR\n",
       "4      48933  tm84618  Cybill Shepherd                    Betsy  ACTOR"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_credits.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评估和清理数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这一部分中，将对在上一部分建立的`original_titles`及`original_credits`DataFrame所包含的数据进行评估和清理。\n",
    "\n",
    "主要从两个方面进行：结构和内容，即整齐度和干净度。\n",
    "\n",
    "数据的结构性问题指不符合“每个变量为一列，每个观察值为一行，每种类型的观察单位为一个表格”这三个标准；数据的内容性问题包括存在丢失数据、重复数据、无效数据等。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了区分开经过清理的数据和原始的数据，创建新的变量`cleaned_titles`，让它为`original_titles`复制出的副本，以及创建新的变量`cleaned_credits`，让它为`original_credits`复制出的变量。之后的清理步骤都将被运用在`cleaned_titles`和`cleaned_credits`上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_titles = original_titles.copy()\n",
    "cleaned_credits = original_credits.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据整齐度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ts300399</td>\n",
       "      <td>Five Came Back: The Reference Films</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>This collection includes 12 World War II-era p...</td>\n",
       "      <td>1945</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>51</td>\n",
       "      <td>['documentation']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.600</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>['drama', 'crime']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>['drama', 'action', 'thriller', 'european']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>['fantasy', 'action', 'comedy']</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>tm120801</td>\n",
       "      <td>The Dirty Dozen</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>12 American military prisoners in World War II...</td>\n",
       "      <td>1967</td>\n",
       "      <td>NaN</td>\n",
       "      <td>150</td>\n",
       "      <td>['war', 'action']</td>\n",
       "      <td>['GB', 'US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0061578</td>\n",
       "      <td>7.7</td>\n",
       "      <td>72662.0</td>\n",
       "      <td>20.398</td>\n",
       "      <td>7.600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ts22164</td>\n",
       "      <td>Monty Python's Flying Circus</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>A British sketch comedy series with the shows ...</td>\n",
       "      <td>1969</td>\n",
       "      <td>TV-14</td>\n",
       "      <td>30</td>\n",
       "      <td>['comedy', 'european']</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>4.0</td>\n",
       "      <td>tt0063929</td>\n",
       "      <td>8.8</td>\n",
       "      <td>73424.0</td>\n",
       "      <td>17.617</td>\n",
       "      <td>8.306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>tm70993</td>\n",
       "      <td>Life of Brian</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Brian Cohen is an average young Jewish man, bu...</td>\n",
       "      <td>1979</td>\n",
       "      <td>R</td>\n",
       "      <td>94</td>\n",
       "      <td>['comedy']</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0079470</td>\n",
       "      <td>8.0</td>\n",
       "      <td>395024.0</td>\n",
       "      <td>17.770</td>\n",
       "      <td>7.800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>tm14873</td>\n",
       "      <td>Dirty Harry</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>When a madman dubbed 'Scorpio' terrorizes San ...</td>\n",
       "      <td>1971</td>\n",
       "      <td>R</td>\n",
       "      <td>102</td>\n",
       "      <td>['thriller', 'action', 'crime']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0066999</td>\n",
       "      <td>7.7</td>\n",
       "      <td>155051.0</td>\n",
       "      <td>12.817</td>\n",
       "      <td>7.500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>tm119281</td>\n",
       "      <td>Bonnie and Clyde</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>In the 1930s, bored waitress Bonnie Parker fal...</td>\n",
       "      <td>1967</td>\n",
       "      <td>R</td>\n",
       "      <td>110</td>\n",
       "      <td>['crime', 'drama', 'action']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0061418</td>\n",
       "      <td>7.7</td>\n",
       "      <td>112048.0</td>\n",
       "      <td>15.687</td>\n",
       "      <td>7.500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>tm98978</td>\n",
       "      <td>The Blue Lagoon</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Two small children and a ship's cook survive a...</td>\n",
       "      <td>1980</td>\n",
       "      <td>R</td>\n",
       "      <td>104</td>\n",
       "      <td>['romance', 'action', 'drama']</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0080453</td>\n",
       "      <td>5.8</td>\n",
       "      <td>69844.0</td>\n",
       "      <td>50.324</td>\n",
       "      <td>6.156</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                title   type  \\\n",
       "0  ts300399  Five Came Back: The Reference Films   SHOW   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "4  tm120801                      The Dirty Dozen  MOVIE   \n",
       "5   ts22164         Monty Python's Flying Circus   SHOW   \n",
       "6   tm70993                        Life of Brian  MOVIE   \n",
       "7   tm14873                          Dirty Harry  MOVIE   \n",
       "8  tm119281                     Bonnie and Clyde  MOVIE   \n",
       "9   tm98978                      The Blue Lagoon  MOVIE   \n",
       "\n",
       "                                         description  release_year  \\\n",
       "0  This collection includes 12 World War II-era p...          1945   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "4  12 American military prisoners in World War II...          1967   \n",
       "5  A British sketch comedy series with the shows ...          1969   \n",
       "6  Brian Cohen is an average young Jewish man, bu...          1979   \n",
       "7  When a madman dubbed 'Scorpio' terrorizes San ...          1971   \n",
       "8  In the 1930s, bored waitress Bonnie Parker fal...          1967   \n",
       "9  Two small children and a ship's cook survive a...          1980   \n",
       "\n",
       "  age_certification  runtime                                       genres  \\\n",
       "0             TV-MA       51                            ['documentation']   \n",
       "1                 R      114                           ['drama', 'crime']   \n",
       "2                 R      109  ['drama', 'action', 'thriller', 'european']   \n",
       "3                PG       91              ['fantasy', 'action', 'comedy']   \n",
       "4               NaN      150                            ['war', 'action']   \n",
       "5             TV-14       30                       ['comedy', 'european']   \n",
       "6                 R       94                                   ['comedy']   \n",
       "7                 R      102              ['thriller', 'action', 'crime']   \n",
       "8                 R      110                 ['crime', 'drama', 'action']   \n",
       "9                 R      104               ['romance', 'action', 'drama']   \n",
       "\n",
       "  production_countries  seasons    imdb_id  imdb_score  imdb_votes  \\\n",
       "0               ['US']      1.0        NaN         NaN         NaN   \n",
       "1               ['US']      NaN  tt0075314         8.2    808582.0   \n",
       "2               ['US']      NaN  tt0068473         7.7    107673.0   \n",
       "3               ['GB']      NaN  tt0071853         8.2    534486.0   \n",
       "4         ['GB', 'US']      NaN  tt0061578         7.7     72662.0   \n",
       "5               ['GB']      4.0  tt0063929         8.8     73424.0   \n",
       "6               ['GB']      NaN  tt0079470         8.0    395024.0   \n",
       "7               ['US']      NaN  tt0066999         7.7    155051.0   \n",
       "8               ['US']      NaN  tt0061418         7.7    112048.0   \n",
       "9               ['US']      NaN  tt0080453         5.8     69844.0   \n",
       "\n",
       "   tmdb_popularity  tmdb_score  \n",
       "0            0.600         NaN  \n",
       "1           40.965       8.179  \n",
       "2           10.010       7.300  \n",
       "3           15.461       7.811  \n",
       "4           20.398       7.600  \n",
       "5           17.617       8.306  \n",
       "6           17.770       7.800  \n",
       "7           12.817       7.500  \n",
       "8           15.687       7.500  \n",
       "9           50.324       6.156  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从数据的部分10行来看，`cleaned_titles`里的`genres`和`production_countries`的变量中包含多个值，应当进行拆分。\n",
    "\n",
    "先提取任意一个`genres`变量的值进行观察。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"['drama', 'crime']\""
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles[\"genres\"][1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然`genres`表示形式是列表，但其实际类型并非字符串列表，而是字符串，无法直接用`value_counts`统计各个值出现的次数。\n",
    "我们可以使用Python内置的`eval`函数，它可以把字符串转换成表达式，所以可以帮我们把表示列表的字符串转换成列表本身。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['drama', 'crime']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles['genres'] = cleaned_titles['genres'].apply(lambda s: eval(s))\n",
    "cleaned_titles['genres'][1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "转换为列表后，就能用DataFrame的`explode`方法，把那个列的列表值拆分成单独的行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ts300399</td>\n",
       "      <td>Five Came Back: The Reference Films</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>This collection includes 12 World War II-era p...</td>\n",
       "      <td>1945</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>51</td>\n",
       "      <td>documentation</td>\n",
       "      <td>['US']</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.600</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>crime</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>drama</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>action</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>thriller</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>european</td>\n",
       "      <td>['US']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>fantasy</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>action</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>comedy</td>\n",
       "      <td>['GB']</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                title   type  \\\n",
       "0  ts300399  Five Came Back: The Reference Films   SHOW   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "\n",
       "                                         description  release_year  \\\n",
       "0  This collection includes 12 World War II-era p...          1945   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "\n",
       "  age_certification  runtime         genres production_countries  seasons  \\\n",
       "0             TV-MA       51  documentation               ['US']      1.0   \n",
       "1                 R      114          drama               ['US']      NaN   \n",
       "1                 R      114          crime               ['US']      NaN   \n",
       "2                 R      109          drama               ['US']      NaN   \n",
       "2                 R      109         action               ['US']      NaN   \n",
       "2                 R      109       thriller               ['US']      NaN   \n",
       "2                 R      109       european               ['US']      NaN   \n",
       "3                PG       91        fantasy               ['GB']      NaN   \n",
       "3                PG       91         action               ['GB']      NaN   \n",
       "3                PG       91         comedy               ['GB']      NaN   \n",
       "\n",
       "     imdb_id  imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "0        NaN         NaN         NaN            0.600         NaN  \n",
       "1  tt0075314         8.2    808582.0           40.965       8.179  \n",
       "1  tt0075314         8.2    808582.0           40.965       8.179  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles = cleaned_titles.explode(\"genres\")\n",
    "cleaned_titles.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，针对`production_countries`列也是一样的流程。\n",
    "\n",
    "每个观察值的`production_countries`值并不表示单个出品国家，而是一系列出品国家。先提取任意一个`production_countries`变量的值进行观察。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    ['US']\n",
       "1    ['US']\n",
       "Name: production_countries, dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles['production_countries'][1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，`production_countries`也是一样的问题，虽然表示形式是列表，但其实际类型并非字符串列表，而是字符串，难以进行拆分。\n",
    "我们可以再次利用`eval`函数进行类型转换，并检查转换后确实是列表类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['US']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles['production_countries'] = cleaned_titles['production_countries'].apply(lambda s: eval(s))\n",
    "cleaned_titles['production_countries'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "确认类型转换完毕后，还是用`explode`方法，把列表值拆分成单独的行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ts300399</td>\n",
       "      <td>Five Came Back: The Reference Films</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>This collection includes 12 World War II-era p...</td>\n",
       "      <td>1945</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>51</td>\n",
       "      <td>documentation</td>\n",
       "      <td>US</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.600</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>crime</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>action</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>thriller</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>european</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>fantasy</td>\n",
       "      <td>GB</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>action</td>\n",
       "      <td>GB</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>tm127384</td>\n",
       "      <td>Monty Python and the Holy Grail</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>King Arthur, accompanied by his squire, recrui...</td>\n",
       "      <td>1975</td>\n",
       "      <td>PG</td>\n",
       "      <td>91</td>\n",
       "      <td>comedy</td>\n",
       "      <td>GB</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0071853</td>\n",
       "      <td>8.2</td>\n",
       "      <td>534486.0</td>\n",
       "      <td>15.461</td>\n",
       "      <td>7.811</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id                                title   type  \\\n",
       "0  ts300399  Five Came Back: The Reference Films   SHOW   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "1   tm84618                          Taxi Driver  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "2  tm154986                          Deliverance  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "3  tm127384      Monty Python and the Holy Grail  MOVIE   \n",
       "\n",
       "                                         description  release_year  \\\n",
       "0  This collection includes 12 World War II-era p...          1945   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "1  A mentally unstable Vietnam War veteran works ...          1976   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "2  Intent on seeing the Cahulawassee River before...          1972   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "3  King Arthur, accompanied by his squire, recrui...          1975   \n",
       "\n",
       "  age_certification  runtime         genres production_countries  seasons  \\\n",
       "0             TV-MA       51  documentation                   US      1.0   \n",
       "1                 R      114          drama                   US      NaN   \n",
       "1                 R      114          crime                   US      NaN   \n",
       "2                 R      109          drama                   US      NaN   \n",
       "2                 R      109         action                   US      NaN   \n",
       "2                 R      109       thriller                   US      NaN   \n",
       "2                 R      109       european                   US      NaN   \n",
       "3                PG       91        fantasy                   GB      NaN   \n",
       "3                PG       91         action                   GB      NaN   \n",
       "3                PG       91         comedy                   GB      NaN   \n",
       "\n",
       "     imdb_id  imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "0        NaN         NaN         NaN            0.600         NaN  \n",
       "1  tt0075314         8.2    808582.0           40.965       8.179  \n",
       "1  tt0075314         8.2    808582.0           40.965       8.179  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2  tt0068473         7.7    107673.0           10.010       7.300  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  \n",
       "3  tt0071853         8.2    534486.0           15.461       7.811  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles = cleaned_titles.explode('production_countries')\n",
    "cleaned_titles.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在处理完`cleaned_titles`的结构性问题后，查看`cleaned_credits`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>character</th>\n",
       "      <th>role</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3748</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Robert De Niro</td>\n",
       "      <td>Travis Bickle</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14658</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Jodie Foster</td>\n",
       "      <td>Iris Steensma</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7064</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Albert Brooks</td>\n",
       "      <td>Tom</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3739</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Harvey Keitel</td>\n",
       "      <td>Matthew 'Sport' Higgins</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>48933</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Cybill Shepherd</td>\n",
       "      <td>Betsy</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>32267</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Peter Boyle</td>\n",
       "      <td>Wizard</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>519612</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Leonard Harris</td>\n",
       "      <td>Senator Charles Palantine</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>29068</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Diahnne Abbott</td>\n",
       "      <td>Concession Girl</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>519613</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Gino Ardito</td>\n",
       "      <td>Policeman at Rally</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>3308</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Martin Scorsese</td>\n",
       "      <td>Passenger Watching Silhouette</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   person_id       id             name                      character   role\n",
       "0       3748  tm84618   Robert De Niro                  Travis Bickle  ACTOR\n",
       "1      14658  tm84618     Jodie Foster                  Iris Steensma  ACTOR\n",
       "2       7064  tm84618    Albert Brooks                            Tom  ACTOR\n",
       "3       3739  tm84618    Harvey Keitel        Matthew 'Sport' Higgins  ACTOR\n",
       "4      48933  tm84618  Cybill Shepherd                          Betsy  ACTOR\n",
       "5      32267  tm84618      Peter Boyle                         Wizard  ACTOR\n",
       "6     519612  tm84618   Leonard Harris      Senator Charles Palantine  ACTOR\n",
       "7      29068  tm84618   Diahnne Abbott                Concession Girl  ACTOR\n",
       "8     519613  tm84618      Gino Ardito             Policeman at Rally  ACTOR\n",
       "9       3308  tm84618  Martin Scorsese  Passenger Watching Silhouette  ACTOR"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_credits.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从头部的10行数据来看，`cleaned_credits`数据符合“每个变量为一列，每个观察值为一行，每种类型的观察单位为一个表格”，因此不存在结构性问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据干净度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来通过`info`，对数据内容进行大致了解。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 17818 entries, 0 to 5849\n",
      "Data columns (total 15 columns):\n",
      " #   Column                Non-Null Count  Dtype  \n",
      "---  ------                --------------  -----  \n",
      " 0   id                    17818 non-null  object \n",
      " 1   title                 17817 non-null  object \n",
      " 2   type                  17818 non-null  object \n",
      " 3   description           17790 non-null  object \n",
      " 4   release_year          17818 non-null  int64  \n",
      " 5   age_certification     10889 non-null  object \n",
      " 6   runtime               17818 non-null  int64  \n",
      " 7   genres                17755 non-null  object \n",
      " 8   production_countries  17439 non-null  object \n",
      " 9   seasons               6224 non-null   float64\n",
      " 10  imdb_id               17116 non-null  object \n",
      " 11  imdb_score            16976 non-null  float64\n",
      " 12  imdb_votes            16945 non-null  float64\n",
      " 13  tmdb_popularity       17663 non-null  float64\n",
      " 14  tmdb_score            17241 non-null  float64\n",
      "dtypes: float64(5), int64(2), object(8)\n",
      "memory usage: 2.2+ MB\n"
     ]
    }
   ],
   "source": [
    "cleaned_titles.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从输出结果来看，`cleaned_titles`数据共有17818条观察值，`title`、`description`、`age_certification`、`genres`、`production_countries`、`seasons`、`imdb_id`、`imdb_score`、`tmdb_popularity`、`tmdb_score`、`imdb_votes`、`tmdb_popularity`、`tmdb_score`变量均存在缺失值，将在后续进行评估和清理。\n",
    "\n",
    "此外，`release_year`表示年份，数据类型不应为数字，应为日期，所以需要进行数据格式转换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      1945-01-01\n",
       "1      1976-01-01\n",
       "1      1976-01-01\n",
       "2      1972-01-01\n",
       "2      1972-01-01\n",
       "          ...    \n",
       "5847   2021-01-01\n",
       "5848   2021-01-01\n",
       "5849   2021-01-01\n",
       "5849   2021-01-01\n",
       "5849   2021-01-01\n",
       "Name: release_year, Length: 17818, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles[\"release_year\"] = pd.to_datetime(cleaned_titles[\"release_year\"], format='%Y')\n",
    "cleaned_titles[\"release_year\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 77801 entries, 0 to 77800\n",
      "Data columns (total 5 columns):\n",
      " #   Column     Non-Null Count  Dtype \n",
      "---  ------     --------------  ----- \n",
      " 0   person_id  77801 non-null  int64 \n",
      " 1   id         77801 non-null  object\n",
      " 2   name       77801 non-null  object\n",
      " 3   character  68029 non-null  object\n",
      " 4   role       77801 non-null  object\n",
      "dtypes: int64(1), object(4)\n",
      "memory usage: 3.0+ MB\n"
     ]
    }
   ],
   "source": [
    "cleaned_credits.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从输出结果来看，`cleaned_credits`数据共有77801条观察值，其中`character`变量存在缺失值，将在后续进行评估和清理。\n",
    "\n",
    "此外，`person_id`表示演职员ID，数据类型不应为数字，应为字符串，所以需要进行数据格式转换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0           3748\n",
       "1          14658\n",
       "2           7064\n",
       "3           3739\n",
       "4          48933\n",
       "          ...   \n",
       "77796     736339\n",
       "77797     399499\n",
       "77798     373198\n",
       "77799     378132\n",
       "77800    1950416\n",
       "Name: person_id, Length: 77801, dtype: object"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_credits[\"person_id\"] = cleaned_credits[\"person_id\"].astype(\"str\")\n",
    "cleaned_credits[\"person_id\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理缺失数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在`cleaned_titles`中，`title`、`description`、`age_certification`、`genres`、`production_countries`、`seasons`、`imdb_id`、`imdb_score`、`tmdb_popularity`、`tmdb_score`、`imdb_votes`、`tmdb_popularity`、`tmdb_score`变量存在缺失值。\n",
    "\n",
    "由于影视作品的标题、描述、适龄认证、发行国家、电视剧季数、IMDB的ID、TMDB的流行度、TMDB的评分，并不影响我们挖掘各个流派中的高IMDB评分作品演员，所以可以保留`title`、`description`、`age_certification`、`production_countries`、`seasons`、`imdb_id`、`tmdb_popularity`、`tmdb_score`、`imdb_votes`、`tmdb_popularity`、`tmdb_score`变量值存在空缺的观察值。\n",
    "\n",
    "但`imdb_score`和`genres`，即IMDB评分和流派，和我们后续要做的分析息息相关。\n",
    "\n",
    "先提取出`imdb_score`缺失观察值进行查看。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ts300399</td>\n",
       "      <td>Five Came Back: The Reference Films</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>This collection includes 12 World War II-era p...</td>\n",
       "      <td>1945-01-01</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>51</td>\n",
       "      <td>documentation</td>\n",
       "      <td>US</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.600</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>tm132164</td>\n",
       "      <td>Bill Hicks: Sane Man</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Sane Man was filmed before Bill recorded ‘Dang...</td>\n",
       "      <td>1989-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>80</td>\n",
       "      <td>comedy</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.377</td>\n",
       "      <td>7.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ts251477</td>\n",
       "      <td>My First Errand</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>“Hajimete no Otsukai” (First Errand) is a Japa...</td>\n",
       "      <td>1991-01-01</td>\n",
       "      <td>TV-G</td>\n",
       "      <td>18</td>\n",
       "      <td>documentation</td>\n",
       "      <td>JP</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.730</td>\n",
       "      <td>7.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ts251477</td>\n",
       "      <td>My First Errand</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>“Hajimete no Otsukai” (First Errand) is a Japa...</td>\n",
       "      <td>1991-01-01</td>\n",
       "      <td>TV-G</td>\n",
       "      <td>18</td>\n",
       "      <td>family</td>\n",
       "      <td>JP</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.730</td>\n",
       "      <td>7.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>ts251477</td>\n",
       "      <td>My First Errand</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>“Hajimete no Otsukai” (First Errand) is a Japa...</td>\n",
       "      <td>1991-01-01</td>\n",
       "      <td>TV-G</td>\n",
       "      <td>18</td>\n",
       "      <td>reality</td>\n",
       "      <td>JP</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.730</td>\n",
       "      <td>7.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5810</th>\n",
       "      <td>tm1225897</td>\n",
       "      <td>Social Man</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Two competitive social media Influencers go he...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>96</td>\n",
       "      <td>drama</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt20198164</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5833</th>\n",
       "      <td>ts307884</td>\n",
       "      <td>HQ Barbers</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>When a family run barber shop in the heart of ...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>TV-14</td>\n",
       "      <td>24</td>\n",
       "      <td>comedy</td>\n",
       "      <td>NG</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.840</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5840</th>\n",
       "      <td>tm1216735</td>\n",
       "      <td>Sun of the Soil</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>In 14th-century Mali, an ambitious young royal...</td>\n",
       "      <td>2022-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.179</td>\n",
       "      <td>7.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5844</th>\n",
       "      <td>tm1074617</td>\n",
       "      <td>Bling Empire - The Afterparty</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>The stars of \"Bling Empire\" discuss the show's...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>35</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5848</th>\n",
       "      <td>tm1035612</td>\n",
       "      <td>Dad Stop Embarrassing Me - The Afterparty</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Jamie Foxx, David Alan Grier and more from the...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>PG-13</td>\n",
       "      <td>37</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.296</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>842 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                      title   type  \\\n",
       "0      ts300399        Five Came Back: The Reference Films   SHOW   \n",
       "75     tm132164                       Bill Hicks: Sane Man  MOVIE   \n",
       "145    ts251477                            My First Errand   SHOW   \n",
       "145    ts251477                            My First Errand   SHOW   \n",
       "145    ts251477                            My First Errand   SHOW   \n",
       "...         ...                                        ...    ...   \n",
       "5810  tm1225897                                 Social Man  MOVIE   \n",
       "5833   ts307884                                 HQ Barbers   SHOW   \n",
       "5840  tm1216735                            Sun of the Soil  MOVIE   \n",
       "5844  tm1074617              Bling Empire - The Afterparty  MOVIE   \n",
       "5848  tm1035612  Dad Stop Embarrassing Me - The Afterparty  MOVIE   \n",
       "\n",
       "                                            description release_year  \\\n",
       "0     This collection includes 12 World War II-era p...   1945-01-01   \n",
       "75    Sane Man was filmed before Bill recorded ‘Dang...   1989-01-01   \n",
       "145   “Hajimete no Otsukai” (First Errand) is a Japa...   1991-01-01   \n",
       "145   “Hajimete no Otsukai” (First Errand) is a Japa...   1991-01-01   \n",
       "145   “Hajimete no Otsukai” (First Errand) is a Japa...   1991-01-01   \n",
       "...                                                 ...          ...   \n",
       "5810  Two competitive social media Influencers go he...   2021-01-01   \n",
       "5833  When a family run barber shop in the heart of ...   2021-01-01   \n",
       "5840  In 14th-century Mali, an ambitious young royal...   2022-01-01   \n",
       "5844  The stars of \"Bling Empire\" discuss the show's...   2021-01-01   \n",
       "5848  Jamie Foxx, David Alan Grier and more from the...   2021-01-01   \n",
       "\n",
       "     age_certification  runtime         genres production_countries  seasons  \\\n",
       "0                TV-MA       51  documentation                   US      1.0   \n",
       "75                   R       80         comedy                   US      NaN   \n",
       "145               TV-G       18  documentation                   JP     12.0   \n",
       "145               TV-G       18         family                   JP     12.0   \n",
       "145               TV-G       18        reality                   JP     12.0   \n",
       "...                ...      ...            ...                  ...      ...   \n",
       "5810               NaN       96          drama                  NaN      NaN   \n",
       "5833             TV-14       24         comedy                   NG      1.0   \n",
       "5840               NaN       26            NaN                  NaN      NaN   \n",
       "5844               NaN       35            NaN                   US      NaN   \n",
       "5848             PG-13       37            NaN                   US      NaN   \n",
       "\n",
       "         imdb_id  imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "0            NaN         NaN         NaN            0.600         NaN  \n",
       "75           NaN         NaN         NaN            3.377         7.5  \n",
       "145          NaN         NaN         NaN            7.730         7.8  \n",
       "145          NaN         NaN         NaN            7.730         7.8  \n",
       "145          NaN         NaN         NaN            7.730         7.8  \n",
       "...          ...         ...         ...              ...         ...  \n",
       "5810  tt20198164         NaN         NaN              NaN         NaN  \n",
       "5833         NaN         NaN         NaN            0.840         NaN  \n",
       "5840         NaN         NaN         NaN            1.179         7.0  \n",
       "5844         NaN         NaN         NaN              NaN         NaN  \n",
       "5848         NaN         NaN         NaN            1.296        10.0  \n",
       "\n",
       "[842 rows x 15 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles.query(\"imdb_score.isnull()\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.int64(0)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles = cleaned_titles.dropna(subset=[\"imdb_score\"])\n",
    "cleaned_titles[\"imdb_score\"].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后提取出`genres`缺失观察值进行查看。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1813</th>\n",
       "      <td>ts77824</td>\n",
       "      <td>My Next Guest Needs No Introduction With David...</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>TV legend David Letterman teams up with fascin...</td>\n",
       "      <td>2018-01-01</td>\n",
       "      <td>TV-MA</td>\n",
       "      <td>50</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>4.0</td>\n",
       "      <td>tt7829834</td>\n",
       "      <td>7.8</td>\n",
       "      <td>5581.0</td>\n",
       "      <td>8.217</td>\n",
       "      <td>7.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1939</th>\n",
       "      <td>ts215037</td>\n",
       "      <td>Minecraft: Story Mode</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>MInecraft: Story Mode is an interactive, anima...</td>\n",
       "      <td>2018-01-01</td>\n",
       "      <td>TV-PG</td>\n",
       "      <td>52</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt10498322</td>\n",
       "      <td>5.6</td>\n",
       "      <td>347.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2386</th>\n",
       "      <td>ts74805</td>\n",
       "      <td>A Little Help with Carol Burnett</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>In this unscripted series starring comedy lege...</td>\n",
       "      <td>2018-01-01</td>\n",
       "      <td>TV-G</td>\n",
       "      <td>24</td>\n",
       "      <td>NaN</td>\n",
       "      <td>US</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt7204366</td>\n",
       "      <td>6.3</td>\n",
       "      <td>237.0</td>\n",
       "      <td>1.621</td>\n",
       "      <td>6.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2658</th>\n",
       "      <td>ts265844</td>\n",
       "      <td>#ABtalks</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>#ABtalks is a YouTube interview show hosted by...</td>\n",
       "      <td>2018-01-01</td>\n",
       "      <td>TV-PG</td>\n",
       "      <td>68</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt12635254</td>\n",
       "      <td>9.6</td>\n",
       "      <td>7.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4274</th>\n",
       "      <td>tm1172010</td>\n",
       "      <td>The Lockdown Plan</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2020-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>49</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt13079112</td>\n",
       "      <td>6.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4648</th>\n",
       "      <td>tm1113921</td>\n",
       "      <td>In Vitro</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>'In Vitro' is an otherworldly rumination on me...</td>\n",
       "      <td>2019-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt10545994</td>\n",
       "      <td>7.7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                                              title   type  \\\n",
       "1813    ts77824  My Next Guest Needs No Introduction With David...   SHOW   \n",
       "1939   ts215037                              Minecraft: Story Mode   SHOW   \n",
       "2386    ts74805                   A Little Help with Carol Burnett   SHOW   \n",
       "2658   ts265844                                           #ABtalks   SHOW   \n",
       "4274  tm1172010                                  The Lockdown Plan  MOVIE   \n",
       "4648  tm1113921                                           In Vitro  MOVIE   \n",
       "\n",
       "                                            description release_year  \\\n",
       "1813  TV legend David Letterman teams up with fascin...   2018-01-01   \n",
       "1939  MInecraft: Story Mode is an interactive, anima...   2018-01-01   \n",
       "2386  In this unscripted series starring comedy lege...   2018-01-01   \n",
       "2658  #ABtalks is a YouTube interview show hosted by...   2018-01-01   \n",
       "4274                                                NaN   2020-01-01   \n",
       "4648  'In Vitro' is an otherworldly rumination on me...   2019-01-01   \n",
       "\n",
       "     age_certification  runtime genres production_countries  seasons  \\\n",
       "1813             TV-MA       50    NaN                   US      4.0   \n",
       "1939             TV-PG       52    NaN                   US      1.0   \n",
       "2386              TV-G       24    NaN                   US      1.0   \n",
       "2658             TV-PG       68    NaN                  NaN      1.0   \n",
       "4274               NaN       49    NaN                  NaN      NaN   \n",
       "4648               NaN       27    NaN                  NaN      NaN   \n",
       "\n",
       "         imdb_id  imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "1813   tt7829834         7.8      5581.0            8.217         7.6  \n",
       "1939  tt10498322         5.6       347.0              NaN         NaN  \n",
       "2386   tt7204366         6.3       237.0            1.621         6.2  \n",
       "2658  tt12635254         9.6         7.0              NaN         NaN  \n",
       "4274  tt13079112         6.5         NaN              NaN         NaN  \n",
       "4648  tt10545994         7.7         NaN              NaN         NaN  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles.query(\"genres.isnull()\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于缺失分析所需的核心数据`genres`，我们将把这些观察值删除，并查看删除后该列空缺值个数和："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.int64(0)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles = cleaned_titles.dropna(subset=[\"genres\"])\n",
    "cleaned_titles[\"genres\"].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来评估`cleaned_credits`的缺失数据，其中只有`character`变量存在缺失值。\n",
    "\n",
    "角色名并不影响我们挖掘各个流派中的高IMDB评分作品演员，并且此变量缺失也有可能因为演职员类别是导演，没有对应角色，因此可以保留`character`变量值存在空缺的观察值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理重复数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据数据变量的含义以及内容来看，`cleaned_titles`里不应该存在每个变量值都相同的观察值，因此查看是否存在重复值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.int64(0)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出结果为0，说明不存在重复值。\n",
    "\n",
    "接下来查看`cleaned_credits`数据表是否存在重复值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "np.int64(0)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_credits.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出结果为0，说明不存在重复值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理不一致数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "针对`cleaned_titles`，不一致数据可能存在于`genres`和`character`变量中，我们将查看是否存在多个不同值指代同一流派，以及多个不同值指代同一国家的情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "genres\n",
       "drama            3357\n",
       "comedy           2419\n",
       "thriller         1446\n",
       "action           1339\n",
       "romance          1080\n",
       "crime            1066\n",
       "documentation     981\n",
       "family            769\n",
       "animation         732\n",
       "fantasy           727\n",
       "european          679\n",
       "scifi             647\n",
       "horror            438\n",
       "history           336\n",
       "music             266\n",
       "reality           226\n",
       "war               221\n",
       "sport             188\n",
       "western            53\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles['genres'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面看出，`genres`列里并不存在不一致数据，各个值都在指代不同的流派。但是里面还存在空字符串表示的流派，并非有效数据，因此可以进行删除。\n",
    "\n",
    "删除后，查看`cleaned_titles`里是否还存在`genres`为空字符串的行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [id, title, type, description, release_year, age_certification, runtime, genres, production_countries, seasons, imdb_id, imdb_score, imdb_votes, tmdb_popularity, tmdb_score]\n",
       "Index: []"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles = cleaned_titles.query('genres != \"\"')\n",
    "cleaned_titles.query('genres == \"\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，针对`production_countries`列也是一样的流程，利用`value_counts`方法，得到`production_countries`的列表里面各个值的出现次数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "production_countries\n",
       "US    5648\n",
       "IN    1610\n",
       "GB    1068\n",
       "JP    1046\n",
       "FR     720\n",
       "      ... \n",
       "CU       1\n",
       "LK       1\n",
       "GT       1\n",
       "AF       1\n",
       "FO       1\n",
       "Name: count, Length: 108, dtype: int64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles['production_countries'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于`value_counts`执行结果中有太多值，Pandas只会默认显示开头和结尾的一些值。要完整展示结果，可以把`display.max_rows`设置为`None`，即取消展示行数上限。\n",
    "\n",
    "但因为我们只是在当前调用`value_counts`时才需要看完整结果，所以可以结合`option_context`，只更改临时上限。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "production_countries\n",
      "US         5648\n",
      "IN         1610\n",
      "GB         1068\n",
      "JP         1046\n",
      "FR          720\n",
      "ES          637\n",
      "KR          637\n",
      "CA          608\n",
      "DE          383\n",
      "CN          295\n",
      "MX          264\n",
      "IT          224\n",
      "BR          221\n",
      "AU          217\n",
      "TR          195\n",
      "PH          192\n",
      "AR          150\n",
      "ID          149\n",
      "BE          148\n",
      "TW          133\n",
      "NG          131\n",
      "PL          126\n",
      "ZA          103\n",
      "HK          102\n",
      "NL          102\n",
      "CO           94\n",
      "EG           93\n",
      "DK           89\n",
      "TH           87\n",
      "SE           81\n",
      "LB           70\n",
      "NO           68\n",
      "AE           52\n",
      "IE           49\n",
      "SG           47\n",
      "XX           43\n",
      "IL           42\n",
      "RU           41\n",
      "CL           35\n",
      "CH           33\n",
      "PS           32\n",
      "BG           31\n",
      "MY           30\n",
      "SA           28\n",
      "IS           28\n",
      "AT           28\n",
      "NZ           27\n",
      "LU           27\n",
      "PE           26\n",
      "RO           25\n",
      "QA           24\n",
      "CZ           22\n",
      "JO           19\n",
      "FI           18\n",
      "HU           18\n",
      "UY           15\n",
      "MA           15\n",
      "PT           14\n",
      "KH           10\n",
      "KW           10\n",
      "PR            9\n",
      "PK            9\n",
      "VN            8\n",
      "UA            8\n",
      "MT            8\n",
      "SU            7\n",
      "TN            7\n",
      "IR            7\n",
      "CD            7\n",
      "LT            7\n",
      "KE            6\n",
      "GH            6\n",
      "AL            6\n",
      "SN            6\n",
      "IQ            5\n",
      "MU            5\n",
      "CY            5\n",
      "TZ            4\n",
      "KN            4\n",
      "SY            4\n",
      "MC            4\n",
      "IO            4\n",
      "GR            4\n",
      "DZ            3\n",
      "HR            3\n",
      "BS            3\n",
      "AO            3\n",
      "GL            3\n",
      "BD            3\n",
      "CM            3\n",
      "PY            3\n",
      "BF            2\n",
      "KG            2\n",
      "BY            2\n",
      "VE            2\n",
      "MW            2\n",
      "RS            2\n",
      "VA            2\n",
      "BT            2\n",
      "ZW            2\n",
      "Lebanon       1\n",
      "NP            1\n",
      "GE            1\n",
      "CU            1\n",
      "LK            1\n",
      "GT            1\n",
      "AF            1\n",
      "FO            1\n",
      "Name: count, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "with pd.option_context('display.max_rows', None):\n",
    "    print(cleaned_titles['production_countries'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从以上输出结果来看，出品国家都用两位的国家代码来表示，除了里面存在一个的`Lebanon`值。\n",
    "\n",
    "`Lebanon`的国家代码是`LB`，出现了39次，说明此处数据不一致。`LB`和`Lebanon`都在表示同一国家，需要进行统一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "把`cleaned_titles`里，`production_countries`的`\"LB\"`和`\"Lebanon\"`统一为`LB`，并检查替换后是否还存在`\"LB\"`："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "production_countries\n",
      "US    5648\n",
      "IN    1610\n",
      "GB    1068\n",
      "JP    1046\n",
      "FR     720\n",
      "ES     637\n",
      "KR     637\n",
      "CA     608\n",
      "DE     383\n",
      "CN     295\n",
      "MX     264\n",
      "IT     224\n",
      "BR     221\n",
      "AU     217\n",
      "TR     195\n",
      "PH     192\n",
      "AR     150\n",
      "ID     149\n",
      "BE     148\n",
      "TW     133\n",
      "NG     131\n",
      "PL     126\n",
      "ZA     103\n",
      "HK     102\n",
      "NL     102\n",
      "CO      94\n",
      "EG      93\n",
      "DK      89\n",
      "TH      87\n",
      "SE      81\n",
      "LB      71\n",
      "NO      68\n",
      "AE      52\n",
      "IE      49\n",
      "SG      47\n",
      "XX      43\n",
      "IL      42\n",
      "RU      41\n",
      "CL      35\n",
      "CH      33\n",
      "PS      32\n",
      "BG      31\n",
      "MY      30\n",
      "SA      28\n",
      "AT      28\n",
      "IS      28\n",
      "NZ      27\n",
      "LU      27\n",
      "PE      26\n",
      "RO      25\n",
      "QA      24\n",
      "CZ      22\n",
      "JO      19\n",
      "FI      18\n",
      "HU      18\n",
      "MA      15\n",
      "UY      15\n",
      "PT      14\n",
      "KW      10\n",
      "KH      10\n",
      "PK       9\n",
      "PR       9\n",
      "UA       8\n",
      "VN       8\n",
      "MT       8\n",
      "SU       7\n",
      "CD       7\n",
      "TN       7\n",
      "LT       7\n",
      "IR       7\n",
      "GH       6\n",
      "SN       6\n",
      "AL       6\n",
      "KE       6\n",
      "IQ       5\n",
      "MU       5\n",
      "CY       5\n",
      "TZ       4\n",
      "SY       4\n",
      "MC       4\n",
      "IO       4\n",
      "KN       4\n",
      "GR       4\n",
      "BD       3\n",
      "BS       3\n",
      "DZ       3\n",
      "GL       3\n",
      "AO       3\n",
      "CM       3\n",
      "HR       3\n",
      "PY       3\n",
      "BF       2\n",
      "VE       2\n",
      "BY       2\n",
      "MW       2\n",
      "RS       2\n",
      "VA       2\n",
      "BT       2\n",
      "ZW       2\n",
      "KG       2\n",
      "NP       1\n",
      "GE       1\n",
      "CU       1\n",
      "LK       1\n",
      "GT       1\n",
      "AF       1\n",
      "FO       1\n",
      "Name: count, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# 对每个观察值\"production_countries\"列的列表运用上面的函数\n",
    "cleaned_titles[\"production_countries\"] = cleaned_titles[\"production_countries\"].replace({\"Lebanon\": \"LB\"})\n",
    "\n",
    "# 检查\"Lebanon\"是否还存在\n",
    "with pd.option_context('display.max_rows', None):\n",
    "    print(cleaned_titles.explode('production_countries')['production_countries'].value_counts())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另外，里面还存在空字符串表示的国家代码，并非有效数据。但由于出品国家并非分析所需的关键信息，所以可以保留出品国家为空的观察值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "针对`original_credits`，不一致数据可能存在于`role`中，我们将查看是否存在多个不同值指代同一演职员类型的情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "role\n",
       "ACTOR       73251\n",
       "DIRECTOR     4550\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_credits['role'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从以上输出结果来看，`role`只有两种可能的值，`ACTOR`或`DIRECTOR`，不存在不一致数据。我们可以把这列的类型转换为`Category`，好处是比字符串类型更节约内存空间，也能表明说值的类型有限。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0           ACTOR\n",
       "1           ACTOR\n",
       "2           ACTOR\n",
       "3           ACTOR\n",
       "4           ACTOR\n",
       "           ...   \n",
       "77796       ACTOR\n",
       "77797       ACTOR\n",
       "77798       ACTOR\n",
       "77799       ACTOR\n",
       "77800    DIRECTOR\n",
       "Name: role, Length: 77801, dtype: category\n",
       "Categories (2, object): ['ACTOR', 'DIRECTOR']"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_credits[\"role\"] = cleaned_credits[\"role\"].astype(\"category\")\n",
    "cleaned_credits[\"role\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 处理无效或错误数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以通过DataFrame的`describe`方法，对数值统计信息进行快速了解。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>release_year</th>\n",
       "      <th>runtime</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5850.000000</td>\n",
       "      <td>5850.000000</td>\n",
       "      <td>2106.000000</td>\n",
       "      <td>5368.000000</td>\n",
       "      <td>5.352000e+03</td>\n",
       "      <td>5759.000000</td>\n",
       "      <td>5539.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2016.417094</td>\n",
       "      <td>76.888889</td>\n",
       "      <td>2.162868</td>\n",
       "      <td>6.510861</td>\n",
       "      <td>2.343938e+04</td>\n",
       "      <td>22.637925</td>\n",
       "      <td>6.829175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>6.937726</td>\n",
       "      <td>39.002509</td>\n",
       "      <td>2.689041</td>\n",
       "      <td>1.163826</td>\n",
       "      <td>9.582047e+04</td>\n",
       "      <td>81.680263</td>\n",
       "      <td>1.170391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1945.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>5.000000e+00</td>\n",
       "      <td>0.009442</td>\n",
       "      <td>0.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2016.000000</td>\n",
       "      <td>44.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>5.800000</td>\n",
       "      <td>5.167500e+02</td>\n",
       "      <td>2.728500</td>\n",
       "      <td>6.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>2018.000000</td>\n",
       "      <td>83.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>6.600000</td>\n",
       "      <td>2.233500e+03</td>\n",
       "      <td>6.821000</td>\n",
       "      <td>6.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2020.000000</td>\n",
       "      <td>104.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>7.300000</td>\n",
       "      <td>9.494000e+03</td>\n",
       "      <td>16.590000</td>\n",
       "      <td>7.537500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2022.000000</td>\n",
       "      <td>240.000000</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>9.600000</td>\n",
       "      <td>2.294231e+06</td>\n",
       "      <td>2274.044000</td>\n",
       "      <td>10.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       release_year      runtime      seasons   imdb_score    imdb_votes  \\\n",
       "count   5850.000000  5850.000000  2106.000000  5368.000000  5.352000e+03   \n",
       "mean    2016.417094    76.888889     2.162868     6.510861  2.343938e+04   \n",
       "std        6.937726    39.002509     2.689041     1.163826  9.582047e+04   \n",
       "min     1945.000000     0.000000     1.000000     1.500000  5.000000e+00   \n",
       "25%     2016.000000    44.000000     1.000000     5.800000  5.167500e+02   \n",
       "50%     2018.000000    83.000000     1.000000     6.600000  2.233500e+03   \n",
       "75%     2020.000000   104.000000     2.000000     7.300000  9.494000e+03   \n",
       "max     2022.000000   240.000000    42.000000     9.600000  2.294231e+06   \n",
       "\n",
       "       tmdb_popularity   tmdb_score  \n",
       "count      5759.000000  5539.000000  \n",
       "mean         22.637925     6.829175  \n",
       "std          81.680263     1.170391  \n",
       "min           0.009442     0.500000  \n",
       "25%           2.728500     6.100000  \n",
       "50%           6.821000     6.900000  \n",
       "75%          16.590000     7.537500  \n",
       "max        2274.044000    10.000000  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "original_titles.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从以上统计信息来看，`original_titles`里不存在脱离现实意义的数值。\n",
    "\n",
    "`original_credits`由于不包含表示数值含义的变量，因此无需用`describe`检查。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 整理数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>tm84618</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>crime</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>action</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>tm154986</td>\n",
       "      <td>Deliverance</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>Intent on seeing the Cahulawassee River before...</td>\n",
       "      <td>1972-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>109</td>\n",
       "      <td>thriller</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0068473</td>\n",
       "      <td>7.7</td>\n",
       "      <td>107673.0</td>\n",
       "      <td>10.010</td>\n",
       "      <td>7.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5846</th>\n",
       "      <td>tm898842</td>\n",
       "      <td>C/O Kaadhal</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A heart warming film that explores the concept...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>134</td>\n",
       "      <td>drama</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt11803618</td>\n",
       "      <td>7.7</td>\n",
       "      <td>348.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5847</th>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Lokillo</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A controversial TV host and comedian who has b...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>90</td>\n",
       "      <td>comedy</td>\n",
       "      <td>CO</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt14585902</td>\n",
       "      <td>3.8</td>\n",
       "      <td>68.0</td>\n",
       "      <td>26.005</td>\n",
       "      <td>6.300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5849</th>\n",
       "      <td>ts271048</td>\n",
       "      <td>Mighty Little Bheem: Kite Festival</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>With winter behind them, Bheem and his townspe...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "      <td>family</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt13711094</td>\n",
       "      <td>7.8</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.289</td>\n",
       "      <td>10.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5849</th>\n",
       "      <td>ts271048</td>\n",
       "      <td>Mighty Little Bheem: Kite Festival</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>With winter behind them, Bheem and his townspe...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "      <td>animation</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt13711094</td>\n",
       "      <td>7.8</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.289</td>\n",
       "      <td>10.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5849</th>\n",
       "      <td>ts271048</td>\n",
       "      <td>Mighty Little Bheem: Kite Festival</td>\n",
       "      <td>SHOW</td>\n",
       "      <td>With winter behind them, Bheem and his townspe...</td>\n",
       "      <td>2021-01-01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "      <td>comedy</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>tt13711094</td>\n",
       "      <td>7.8</td>\n",
       "      <td>18.0</td>\n",
       "      <td>2.289</td>\n",
       "      <td>10.000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>16970 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             id                               title   type  \\\n",
       "1       tm84618                         Taxi Driver  MOVIE   \n",
       "1       tm84618                         Taxi Driver  MOVIE   \n",
       "2      tm154986                         Deliverance  MOVIE   \n",
       "2      tm154986                         Deliverance  MOVIE   \n",
       "2      tm154986                         Deliverance  MOVIE   \n",
       "...         ...                                 ...    ...   \n",
       "5846   tm898842                         C/O Kaadhal  MOVIE   \n",
       "5847  tm1059008                             Lokillo  MOVIE   \n",
       "5849   ts271048  Mighty Little Bheem: Kite Festival   SHOW   \n",
       "5849   ts271048  Mighty Little Bheem: Kite Festival   SHOW   \n",
       "5849   ts271048  Mighty Little Bheem: Kite Festival   SHOW   \n",
       "\n",
       "                                            description release_year  \\\n",
       "1     A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "1     A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "2     Intent on seeing the Cahulawassee River before...   1972-01-01   \n",
       "2     Intent on seeing the Cahulawassee River before...   1972-01-01   \n",
       "2     Intent on seeing the Cahulawassee River before...   1972-01-01   \n",
       "...                                                 ...          ...   \n",
       "5846  A heart warming film that explores the concept...   2021-01-01   \n",
       "5847  A controversial TV host and comedian who has b...   2021-01-01   \n",
       "5849  With winter behind them, Bheem and his townspe...   2021-01-01   \n",
       "5849  With winter behind them, Bheem and his townspe...   2021-01-01   \n",
       "5849  With winter behind them, Bheem and his townspe...   2021-01-01   \n",
       "\n",
       "     age_certification  runtime     genres production_countries  seasons  \\\n",
       "1                    R      114      drama                   US      NaN   \n",
       "1                    R      114      crime                   US      NaN   \n",
       "2                    R      109      drama                   US      NaN   \n",
       "2                    R      109     action                   US      NaN   \n",
       "2                    R      109   thriller                   US      NaN   \n",
       "...                ...      ...        ...                  ...      ...   \n",
       "5846               NaN      134      drama                  NaN      NaN   \n",
       "5847               NaN       90     comedy                   CO      NaN   \n",
       "5849               NaN        7     family                  NaN      1.0   \n",
       "5849               NaN        7  animation                  NaN      1.0   \n",
       "5849               NaN        7     comedy                  NaN      1.0   \n",
       "\n",
       "         imdb_id  imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "1      tt0075314         8.2    808582.0           40.965       8.179  \n",
       "1      tt0075314         8.2    808582.0           40.965       8.179  \n",
       "2      tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2      tt0068473         7.7    107673.0           10.010       7.300  \n",
       "2      tt0068473         7.7    107673.0           10.010       7.300  \n",
       "...          ...         ...         ...              ...         ...  \n",
       "5846  tt11803618         7.7       348.0              NaN         NaN  \n",
       "5847  tt14585902         3.8        68.0           26.005       6.300  \n",
       "5849  tt13711094         7.8        18.0            2.289      10.000  \n",
       "5849  tt13711094         7.8        18.0            2.289      10.000  \n",
       "5849  tt13711094         7.8        18.0            2.289      10.000  \n",
       "\n",
       "[16970 rows x 15 columns]"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_titles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>character</th>\n",
       "      <th>role</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3748</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Robert De Niro</td>\n",
       "      <td>Travis Bickle</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14658</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Jodie Foster</td>\n",
       "      <td>Iris Steensma</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7064</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Albert Brooks</td>\n",
       "      <td>Tom</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3739</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Harvey Keitel</td>\n",
       "      <td>Matthew 'Sport' Higgins</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>48933</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Cybill Shepherd</td>\n",
       "      <td>Betsy</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77796</th>\n",
       "      <td>736339</td>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Adelaida Buscato</td>\n",
       "      <td>María Paz</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77797</th>\n",
       "      <td>399499</td>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Luz Stella Luengas</td>\n",
       "      <td>Karen Bayona</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77798</th>\n",
       "      <td>373198</td>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Inés Prieto</td>\n",
       "      <td>Fanny</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77799</th>\n",
       "      <td>378132</td>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Isabel Gaona</td>\n",
       "      <td>Cacica</td>\n",
       "      <td>ACTOR</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77800</th>\n",
       "      <td>1950416</td>\n",
       "      <td>tm1059008</td>\n",
       "      <td>Julian Gaviria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>DIRECTOR</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>77801 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      person_id         id                name                character  \\\n",
       "0          3748    tm84618      Robert De Niro            Travis Bickle   \n",
       "1         14658    tm84618        Jodie Foster            Iris Steensma   \n",
       "2          7064    tm84618       Albert Brooks                      Tom   \n",
       "3          3739    tm84618       Harvey Keitel  Matthew 'Sport' Higgins   \n",
       "4         48933    tm84618     Cybill Shepherd                    Betsy   \n",
       "...         ...        ...                 ...                      ...   \n",
       "77796    736339  tm1059008    Adelaida Buscato                María Paz   \n",
       "77797    399499  tm1059008  Luz Stella Luengas             Karen Bayona   \n",
       "77798    373198  tm1059008         Inés Prieto                    Fanny   \n",
       "77799    378132  tm1059008        Isabel Gaona                   Cacica   \n",
       "77800   1950416  tm1059008      Julian Gaviria                      NaN   \n",
       "\n",
       "           role  \n",
       "0         ACTOR  \n",
       "1         ACTOR  \n",
       "2         ACTOR  \n",
       "3         ACTOR  \n",
       "4         ACTOR  \n",
       "...         ...  \n",
       "77796     ACTOR  \n",
       "77797     ACTOR  \n",
       "77798     ACTOR  \n",
       "77799     ACTOR  \n",
       "77800  DIRECTOR  \n",
       "\n",
       "[77801 rows x 5 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cleaned_credits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对数据的整理，与分析方向紧密相关。此次数据分析目标是，整理不同流派影视作品，比如喜剧片、动作片、科幻片中，演员出演作品的平均IMDB评分，从而挖掘出各个流派中的高评分作品演员。\n",
    "\n",
    "那为了能同时获得流派与演员数据，我们需要把`cleaned_titles`和`cleaned_credits`，通过`id`作为键进行连接，因为两个数据表中`id`都是影视作品ID。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "credits_with_titles = pd.merge(cleaned_credits,cleaned_titles,on=\"id\",how=\"inner\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "连接后，我们就能知道各个演职员参与过的影视作品的具体信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>id</th>\n",
       "      <th>name</th>\n",
       "      <th>character</th>\n",
       "      <th>role</th>\n",
       "      <th>title</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>release_year</th>\n",
       "      <th>age_certification</th>\n",
       "      <th>runtime</th>\n",
       "      <th>genres</th>\n",
       "      <th>production_countries</th>\n",
       "      <th>seasons</th>\n",
       "      <th>imdb_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>imdb_votes</th>\n",
       "      <th>tmdb_popularity</th>\n",
       "      <th>tmdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3748</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Robert De Niro</td>\n",
       "      <td>Travis Bickle</td>\n",
       "      <td>ACTOR</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3748</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Robert De Niro</td>\n",
       "      <td>Travis Bickle</td>\n",
       "      <td>ACTOR</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>crime</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>14658</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Jodie Foster</td>\n",
       "      <td>Iris Steensma</td>\n",
       "      <td>ACTOR</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>14658</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Jodie Foster</td>\n",
       "      <td>Iris Steensma</td>\n",
       "      <td>ACTOR</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>crime</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7064</td>\n",
       "      <td>tm84618</td>\n",
       "      <td>Albert Brooks</td>\n",
       "      <td>Tom</td>\n",
       "      <td>ACTOR</td>\n",
       "      <td>Taxi Driver</td>\n",
       "      <td>MOVIE</td>\n",
       "      <td>A mentally unstable Vietnam War veteran works ...</td>\n",
       "      <td>1976-01-01</td>\n",
       "      <td>R</td>\n",
       "      <td>114</td>\n",
       "      <td>drama</td>\n",
       "      <td>US</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tt0075314</td>\n",
       "      <td>8.2</td>\n",
       "      <td>808582.0</td>\n",
       "      <td>40.965</td>\n",
       "      <td>8.179</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  person_id       id            name      character   role        title  \\\n",
       "0      3748  tm84618  Robert De Niro  Travis Bickle  ACTOR  Taxi Driver   \n",
       "1      3748  tm84618  Robert De Niro  Travis Bickle  ACTOR  Taxi Driver   \n",
       "2     14658  tm84618    Jodie Foster  Iris Steensma  ACTOR  Taxi Driver   \n",
       "3     14658  tm84618    Jodie Foster  Iris Steensma  ACTOR  Taxi Driver   \n",
       "4      7064  tm84618   Albert Brooks            Tom  ACTOR  Taxi Driver   \n",
       "\n",
       "    type                                        description release_year  \\\n",
       "0  MOVIE  A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "1  MOVIE  A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "2  MOVIE  A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "3  MOVIE  A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "4  MOVIE  A mentally unstable Vietnam War veteran works ...   1976-01-01   \n",
       "\n",
       "  age_certification  runtime genres production_countries  seasons    imdb_id  \\\n",
       "0                 R      114  drama                   US      NaN  tt0075314   \n",
       "1                 R      114  crime                   US      NaN  tt0075314   \n",
       "2                 R      114  drama                   US      NaN  tt0075314   \n",
       "3                 R      114  crime                   US      NaN  tt0075314   \n",
       "4                 R      114  drama                   US      NaN  tt0075314   \n",
       "\n",
       "   imdb_score  imdb_votes  tmdb_popularity  tmdb_score  \n",
       "0         8.2    808582.0           40.965       8.179  \n",
       "1         8.2    808582.0           40.965       8.179  \n",
       "2         8.2    808582.0           40.965       8.179  \n",
       "3         8.2    808582.0           40.965       8.179  \n",
       "4         8.2    808582.0           40.965       8.179  "
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "credits_with_titles.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于我们只对挖掘演员的参演作品口碑感兴趣，导演不在我们的分析范围内，因此根据`role`，筛选出类型为`ACTOR`的观察值，供后续分析。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "actor_with_titles = credits_with_titles.query(\"role =='ACTOR'\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了挖掘出各个流派中的高IMDB评分作品演员，我们需要先根据流派和演员进行分组。\n",
    "\n",
    "对演员进行分组的时候，选择的是用`person_id`而不是`name`变量，原因是名字容易出现错拼或者重名的情况，演职员ID会比演员姓名更加准确地反映是哪位演员。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "groupby_genres_and_person_id = actor_with_titles.groupby([\"genres\",\"person_id\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "genres   person_id\n",
       "action   1000         6.866667\n",
       "         100007       7.000000\n",
       "         100013       6.400000\n",
       "         100019       6.500000\n",
       "         100020       6.500000\n",
       "                        ...   \n",
       "western  993735       6.500000\n",
       "         998673       7.300000\n",
       "         998674       7.300000\n",
       "         998675       7.300000\n",
       "         99940        4.000000\n",
       "Name: imdb_score, Length: 168881, dtype: float64"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_score_groupby_genres_and_person_id = groupby_genres_and_person_id[\"imdb_score\"].mean()\n",
    "imdb_score_groupby_genres_and_person_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以调用`reset_index`，对层次化索引进行重置，得到更加规整的DataFrame。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>person_id</th>\n",
       "      <th>imdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>action</td>\n",
       "      <td>1000</td>\n",
       "      <td>6.866667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>action</td>\n",
       "      <td>100007</td>\n",
       "      <td>7.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>action</td>\n",
       "      <td>100013</td>\n",
       "      <td>6.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>action</td>\n",
       "      <td>100019</td>\n",
       "      <td>6.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>action</td>\n",
       "      <td>100020</td>\n",
       "      <td>6.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168876</th>\n",
       "      <td>western</td>\n",
       "      <td>993735</td>\n",
       "      <td>6.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168877</th>\n",
       "      <td>western</td>\n",
       "      <td>998673</td>\n",
       "      <td>7.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168878</th>\n",
       "      <td>western</td>\n",
       "      <td>998674</td>\n",
       "      <td>7.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168879</th>\n",
       "      <td>western</td>\n",
       "      <td>998675</td>\n",
       "      <td>7.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168880</th>\n",
       "      <td>western</td>\n",
       "      <td>99940</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>168881 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         genres person_id  imdb_score\n",
       "0        action      1000    6.866667\n",
       "1        action    100007    7.000000\n",
       "2        action    100013    6.400000\n",
       "3        action    100019    6.500000\n",
       "4        action    100020    6.500000\n",
       "...         ...       ...         ...\n",
       "168876  western    993735    6.500000\n",
       "168877  western    998673    7.300000\n",
       "168878  western    998674    7.300000\n",
       "168879  western    998675    7.300000\n",
       "168880  western     99940    4.000000\n",
       "\n",
       "[168881 rows x 3 columns]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_score_groupby_genres_and_person_id_df = imdb_score_groupby_genres_and_person_id.reset_index()\n",
    "imdb_score_groupby_genres_and_person_id_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在针对流派和演员分组的IMDB评分数据已经整理好，可以进入后续的分析步骤了。\n",
    "\n",
    "但我们当前可以继续做一些数据整理，比如对上面的结果再次进行分组，找出各个流派里演员作品最高的平均评分是多少、最高评分对应的演员名字是什么。\n",
    "\n",
    "要得到这一结果，我们需要再次用`genres`进行分组，然后提取出`imdb_score`变量，计算其最大值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "genres\n",
       "action           9.3\n",
       "animation        9.3\n",
       "comedy           9.2\n",
       "crime            9.5\n",
       "documentation    9.1\n",
       "drama            9.5\n",
       "european         8.9\n",
       "family           9.3\n",
       "fantasy          9.3\n",
       "history          9.1\n",
       "horror           9.0\n",
       "music            8.8\n",
       "reality          8.9\n",
       "romance          9.2\n",
       "scifi            9.3\n",
       "sport            9.1\n",
       "thriller         9.5\n",
       "war              8.8\n",
       "western          8.9\n",
       "Name: imdb_score, dtype: float64"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genres_max_scores = imdb_score_groupby_genres_and_person_id_df.groupby(\"genres\")[\"imdb_score\"].max()\n",
    "genres_max_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在我们知道最高分后，可以把以上结果和之前得到的`imdb_score_groupby_genres_and_person_id_df`再次进行连接，得到最高分对应的各个演员ID是什么，也就是这个最高平均分是哪位演员拿到的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>person_id</th>\n",
       "      <th>imdb_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>action</td>\n",
       "      <td>12790</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>action</td>\n",
       "      <td>1303</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>action</td>\n",
       "      <td>21033</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>action</td>\n",
       "      <td>336830</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>action</td>\n",
       "      <td>86591</td>\n",
       "      <td>9.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>war</td>\n",
       "      <td>826547</td>\n",
       "      <td>8.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>western</td>\n",
       "      <td>22311</td>\n",
       "      <td>8.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>western</td>\n",
       "      <td>28166</td>\n",
       "      <td>8.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>western</td>\n",
       "      <td>28180</td>\n",
       "      <td>8.9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>western</td>\n",
       "      <td>93017</td>\n",
       "      <td>8.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>136 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      genres person_id  imdb_score\n",
       "0     action     12790         9.3\n",
       "1     action      1303         9.3\n",
       "2     action     21033         9.3\n",
       "3     action    336830         9.3\n",
       "4     action     86591         9.3\n",
       "..       ...       ...         ...\n",
       "131      war    826547         8.8\n",
       "132  western     22311         8.9\n",
       "133  western     28166         8.9\n",
       "134  western     28180         8.9\n",
       "135  western     93017         8.9\n",
       "\n",
       "[136 rows x 3 columns]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genres_max_score_with_person_id = pd.merge(imdb_score_groupby_genres_and_person_id_df,genres_max_scores,on=[\"genres\",\"imdb_score\"])\n",
    "genres_max_score_with_person_id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从以上结果可以看出，最高分对应的演员不一定只有一位，可能有多位演员的平均得分相同。\n",
    "\n",
    "为了得到演员ID所对应的演员名字，我们可以和`cleaned_credits`这个DataFrame进行连接。这个DataFrame还有其它列，我们只需要得到`person_id`和`name`的对应，所以可以先提取出那两列，并把重复行删除。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person_id</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3748</td>\n",
       "      <td>Robert De Niro</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14658</td>\n",
       "      <td>Jodie Foster</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7064</td>\n",
       "      <td>Albert Brooks</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3739</td>\n",
       "      <td>Harvey Keitel</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>48933</td>\n",
       "      <td>Cybill Shepherd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>32267</td>\n",
       "      <td>Peter Boyle</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>519612</td>\n",
       "      <td>Leonard Harris</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>29068</td>\n",
       "      <td>Diahnne Abbott</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>519613</td>\n",
       "      <td>Gino Ardito</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>3308</td>\n",
       "      <td>Martin Scorsese</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  person_id             name\n",
       "0      3748   Robert De Niro\n",
       "1     14658     Jodie Foster\n",
       "2      7064    Albert Brooks\n",
       "3      3739    Harvey Keitel\n",
       "4     48933  Cybill Shepherd\n",
       "5     32267      Peter Boyle\n",
       "6    519612   Leonard Harris\n",
       "7     29068   Diahnne Abbott\n",
       "8    519613      Gino Ardito\n",
       "9      3308  Martin Scorsese"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "actor_id_with_names = cleaned_credits[['person_id', 'name']].drop_duplicates()\n",
    "actor_id_with_names.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一步就可以把`actor_id_with_names`与前面得到的`genres_max_score_with_person_id`进行连接，增加`name`变量，从而展示平均评分最高的演员名字。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>person_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>action</td>\n",
       "      <td>12790</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Olivia Hack</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>action</td>\n",
       "      <td>1303</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Jessie Flower</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>action</td>\n",
       "      <td>21033</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Zach Tyler</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>action</td>\n",
       "      <td>336830</td>\n",
       "      <td>9.3</td>\n",
       "      <td>André Sogliuzzo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>action</td>\n",
       "      <td>86591</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Cricket Leigh</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>war</td>\n",
       "      <td>826547</td>\n",
       "      <td>8.8</td>\n",
       "      <td>Yuto Uemura</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>western</td>\n",
       "      <td>22311</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Koichi Yamadera</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>western</td>\n",
       "      <td>28166</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Megumi Hayashibara</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>western</td>\n",
       "      <td>28180</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Unsho Ishizuka</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>western</td>\n",
       "      <td>93017</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Aoi Tada</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>136 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      genres person_id  imdb_score                name\n",
       "0     action     12790         9.3         Olivia Hack\n",
       "1     action      1303         9.3       Jessie Flower\n",
       "2     action     21033         9.3          Zach Tyler\n",
       "3     action    336830         9.3     André Sogliuzzo\n",
       "4     action     86591         9.3       Cricket Leigh\n",
       "..       ...       ...         ...                 ...\n",
       "131      war    826547         8.8         Yuto Uemura\n",
       "132  western     22311         8.9     Koichi Yamadera\n",
       "133  western     28166         8.9  Megumi Hayashibara\n",
       "134  western     28180         8.9      Unsho Ishizuka\n",
       "135  western     93017         8.9            Aoi Tada\n",
       "\n",
       "[136 rows x 4 columns]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genres_max_score_with_actor_name = pd.merge(genres_max_score_with_person_id, actor_id_with_names, on=\"person_id\")\n",
    "genres_max_score_with_actor_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了把相同流派都排序在一起，我们还可以用`sort_values`方法，把结果里面的行根据`genres`进行排序，然后用`reset_index`把索引重新排序。\n",
    "\n",
    "索引重新排序后，DataFrame会多出`index`一列，我们可以再把`index`列进行删除。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genres</th>\n",
       "      <th>person_id</th>\n",
       "      <th>imdb_score</th>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>action</td>\n",
       "      <td>12790</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Olivia Hack</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>action</td>\n",
       "      <td>1303</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Jessie Flower</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>action</td>\n",
       "      <td>21033</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Zach Tyler</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>action</td>\n",
       "      <td>336830</td>\n",
       "      <td>9.3</td>\n",
       "      <td>André Sogliuzzo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>action</td>\n",
       "      <td>86591</td>\n",
       "      <td>9.3</td>\n",
       "      <td>Cricket Leigh</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131</th>\n",
       "      <td>war</td>\n",
       "      <td>826547</td>\n",
       "      <td>8.8</td>\n",
       "      <td>Yuto Uemura</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132</th>\n",
       "      <td>western</td>\n",
       "      <td>28180</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Unsho Ishizuka</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>western</td>\n",
       "      <td>22311</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Koichi Yamadera</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>134</th>\n",
       "      <td>western</td>\n",
       "      <td>28166</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Megumi Hayashibara</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>135</th>\n",
       "      <td>western</td>\n",
       "      <td>93017</td>\n",
       "      <td>8.9</td>\n",
       "      <td>Aoi Tada</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>136 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      genres person_id  imdb_score                name\n",
       "0     action     12790         9.3         Olivia Hack\n",
       "1     action      1303         9.3       Jessie Flower\n",
       "2     action     21033         9.3          Zach Tyler\n",
       "3     action    336830         9.3     André Sogliuzzo\n",
       "4     action     86591         9.3       Cricket Leigh\n",
       "..       ...       ...         ...                 ...\n",
       "131      war    826547         8.8         Yuto Uemura\n",
       "132  western     28180         8.9      Unsho Ishizuka\n",
       "133  western     22311         8.9     Koichi Yamadera\n",
       "134  western     28166         8.9  Megumi Hayashibara\n",
       "135  western     93017         8.9            Aoi Tada\n",
       "\n",
       "[136 rows x 4 columns]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "genres_max_score_with_actor_name = genres_max_score_with_actor_name.sort_values(\"genres\").reset_index().drop(\"index\", axis=1)\n",
    "genres_max_score_with_actor_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
